A Fast Algorithm for Separated Sparsity via Perturbed Lagrangians

Sparsity-based methods are widely used in machine learning, statistics, and signal processing. Thereis now a rich class of structured sparsity approaches that expand the modeling power of the sparsityparadigm and incorporate constraints such as group sparsity, graph sparsity, or hierarchical sparsity. Whilethese sparsity models offer improved sample complexity and better interpretability, the improvementscome at a computational cost: it is often challenging to optimize over the (non-convex) constraint setsthat capture various sparsity structures. In this paper, we make progress in this direction in the context ofseparated sparsity – a fundamental sparsity notion that captures exclusion constraints in linearly ordereddata such as time series. While prior algorithms for computing a projection onto this constraint setrequired quadratic time, we provide a perturbed Lagrangian relaxation approach that computes provablyexact projection in only nearly-linear time. Although the sparsity constraint is non-convex, our perturbedLagrangian approach is still guaranteed to find a globally optimal solution. In experiments, our newalgorithms offer a 10× speed-up already on moderately-size inputs. 1arXiv:1712.08130v1[cs.DS]21Dec2017